Omissions are a major risk in testing. Test planning techniques are a well-known way to improve the quality of testing by reducing omissions, redundancy and execution time. These techniques try to prevent omissions in advance (i.e., before tests are implemented and executed) by selecting what to test out of a possibly enormous test space, in a way that reduces as much as possible the risk of bugs escaping to the field. The selection of tests is also characterized in avoiding or reducing tests that are similar to one another and provide no added value. In some cases, three tests may have the same coverage as two tests, as the third test may overlap both two tests (e.g., a first part overlaps the first test and the rest of the test overlaps the second test).
For the purpose of testing, a model of the test space may be defined. The test space can represent inputs, scenarios, configurations, or any other aspect that one is interested in testing. The model comprises a set of functional attributes, respective domains of values for each attribute, and potentially restrictions on the value combinations. As an example, the test space may be defined by inputs formed as functional attributes and its values. This model may be used to determine the description of a test plan. The description may comprise coverage tasks, each having a value assigned to each functional attribute, and representing different setup activities to be performed prior to executing each test.
In some cases, Combinatorial Test Design (CTD) may be used to create a test plan based on a model which covers a desired coverage goal, which may be defined as a target level of interaction between the attributes. Additionally or alternatively, the test plan may be created manually, may be generated automatically using other techniques different than CTD, may be defined based on an existing test suite, or the like.